Computers are only as useful, generally, as the programs that the computer will run. Writing software for these programs has become more efficient in the last few years. One area of improvement is the tools and technologies that address one particular component of the program construction, namely, incremental program development (IPD). There are at least two separate aspects to IPD. One is providing a tight loop of edit, build, compile, link, and debug when modifying existing programs another is providing support for rapid prototyping when developing or testing new functionality.
When developing new functionality, it is often desirable to be able to execute partial program, individual functions, or even single lines of code, in order to test the functionality. This capability is also referred to as unit testing, although the latter term can also apply to regression testing of finished code. Typical debuggers all provide the means to execute individual functions in the program being debugged, or to evaluate expressions in the context of the debugged program, but these actions require the presence of a fully linked executable.
When developing new code, the programmer must supply stubs for any functions or objects that have not yet been implemented, and in order to effect any change to the new code being tested, the entire edit-build-compile-link-debug loop must be executed. While providing support for a tight loop would help, what is really needed is the capability to instantly see the effects of a given piece of code independent of the rest of the program.
Incremental program development consists of both the tight loop and rapid prototyping. In order to accelerate the tight loop, we can provide incremental linkers, incremental compilers, and better interaction between debuggers and programs being debugged. In order to support rapid prototyping we can provide a more transparent interface between the debugger and the program being debugged, and provide an interpreter than is integrated with such a debugger.
Tight Loop
The standard Unix edit-build-compile-link-debug loop is cumbersome and slow, especially when used to maintain large programs. Current technology helps only in the edit phase, by blurring the line between source files and providing a coherent view of the whole program, through tools such as SoftStatic, a Hewlett-Packard Company product. But when a change is made to the source, the build phase still analyzes all dependencies for the entire program to determine what needs to be recompiled, the compile phase still recompiles the entire source file that was modified, the link phase still recombines all object files and libraries to form the program, and the debugger must still be reloaded with the new program.
The edit phase is inherently incremental in that only the files that are to be changed need to be processed by the editor although it could be made more incremental if only the section of code that is altered would be read or written. The compile phase is inherently incremental in that only the files that were modified or were affected by the modifications are processed by the compiler, although it could be made more incremental if only the affected functions or objects were recompiled. The build, link, and debug phases are entirely monolithic. Any change to a program, no matter how small, causes a full dependency analysis by the build tool, a full link by the linker, and a full program reload by the debugger.
Incremental Linking
A linker combines the object files and libraries that comprise a program into a single executable file. It concatenates the individual text and data segments of the input files, relocating the segments as they are copied. When a change is made to a single object file, the entire program must be relinked. All object files and libraries, even those unaffected by the change, are concatenated again to form a new executable file.
Static Linking
One way to introduce incremental linking into the relinking process is to have the linker modify the original executable itself, rather than regenerating it from its constituent object files and libraries. The new code and data could be overlayed on top of the original versions, or placed in areas that were specially reserved to hold new code and data. All references to the replaced code and data would have to be patched to refer to the new versions.
There are two problems with this type of scheme. First, it does not scale well. When a change is made to a function such that it no longer fits in the space allocated for it, the executable file must be regenerated. A prototype incremental liner has been developed at Stanford, and this program has a considerable amount of complexity devoted to management of space in the executable file. When space allocated for new code and data is exhausted, the entire executable file from that point on is rewritten. While the claim is that this happens rarely, it has only been tested on relatively small programs, and only on relatively finished programs that experienced only small changes.
The second problem is that it is not very portable, and, as more complexity is added to object file formats, it becomes difficult to support even on a single architecture. For example, the three object file formats that some linkers would need to support are radically different. An incremental linker based on the Stanford design does not handle the various tables used to support shared libraries, and could not be modified to do so without considerable effort. For example, none of this work would apply directly to the Hewlett-Packard Series 700, which has an entirely different object file format, including different symbol table format, relocation types, and shared library tables. Also, none of the work would be applicable to sunOS 4.1 and then much of any SunOS 4.1 implementation would have to be rewritten to work on Solaris, which will feature an entirely new object file format.
One alternative that alleviates these problems is to develop a simple, portable format for executable files that allows new code and data to be easily appended. A special loader would have to be developed as well. It is not certain how easy it would be to abstract away the various complexities in object file formats such as SOM. It is also uncertain what would be the performance implications of using a non-native loader. A simple portable format for executable files is not necessarily a good solution because object file formats differ greatly, and special loaders would probably be slow.
Centerline
Centerline provides both a C and C++ development environment based on an integrated interpreter and debugger that also allows for dynamic loading of compiled code. The interpreter allows for rapid prototyping and a fight loop when in maintenance, and the incremental linking functionality, in the form of a dynamic loader, makes the system usable even for relatively large programs, as long as the working set of modules being interpreted is relatively small. One of the biggest weaknesses of Centerline as a tight loop or rapid prototyping environment is that no compiler is provided at all, so the development environment is completely isolated from the eventual build environment. Performance concerns encourage the user to compile as much code as possible, but in order to gain full advantage of the Centerline environment, code must be interpreted.
Lucid
The Lucid Cadillac system has particular features that support incremental program development, an incremental compiler and incremental linker and loader. When a change is made to a source file, only the functions and objects that were affected are recompiled by the system and the resultant incremental compilation unit is incrementally linked into the running program as well as the existing executable file. In theory, this should allow for a fast tight loop, including the ability to continue executing the program in the debugger where it left off. However, people who have seen demos of Cadillac have reported turnaround times are not actually particularly fast, nor can a program continue in the debugger after being modified.